Skip to content

[Synthetics] Add synthetics/api input + api / api.network data streams#19178

Draft
shahzad31 wants to merge 2 commits into
mainfrom
synthetics-api-journey
Draft

[Synthetics] Add synthetics/api input + api / api.network data streams#19178
shahzad31 wants to merge 2 commits into
mainfrom
synthetics-api-journey

Conversation

@shahzad31

@shahzad31 shahzad31 commented May 23, 2026

Copy link
Copy Markdown
Contributor

Proposed commit message

Adds the Fleet integration package side of API Journey monitors so private locations can deliver monitor.type: api policies to Heartbeat. Companion to:

The Kibana PR has a TODO in format_synthetics_policy.ts: inputs.find(input.type === 'synthetics/api') resolves to undefined until a synthetics >= 1.7.0 package is installed, so private-location API monitors return hasInput: false and surface a "package needs upgrade" warning. That PR also explicitly says the new input must be designed fresh, not forked from synthetics/browser, because the var schema differs.

What's in this PR

  • manifest.yml: bump 1.6.1 → 1.7.0 and add synthetics/api input.

  • New data stream api (dataset api) — journey/step events for API monitors. ILM 30d hot / 365d delete (matches browser). Vars are the API-correct subset of browser:

  • New data stream api_network (dataset api.network) — the dataset synthexec/enrich.go in feat(heartbeat): add api monitor type for synthetics API journeys beats#50802 routes API-journey journey/network_info events to. ILM 1d hot / 14d delete (matches browser_network). Fields mirror browser_network (TLS / server / resolve groups already covered by ECS) with the same journey.type addition. No stream-level vars — inherits from the parent synthetics/api input, same model as browser_network.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current.
  • N/A — no dashboards added.

How to test this PR locally

cd packages/synthetics
elastic-package format
elastic-package check       # lint + build, both must be green

End-to-end (requires the companion PRs):

  1. Build a Heartbeat with feat(heartbeat): add api monitor type for synthetics API journeys beats#50802 and a @elastic/synthetics from feat: multi step api runner synthetics#997.
  2. Build a Kibana with [Synthetics] Add API Journey monitor type !! kibana#270874 and install this synthetics-1.7.0 package.
  3. Create an API Journey monitor in the Synthetics UI with a private location.
  4. Verify the Fleet policy lands with inputs[].type: synthetics/api enabled and Heartbeat executes the journey.
  5. Verify journey/step events flow into synthetics-api-default and per-request network events flow into synthetics-api.network-default.

Related issues

Out of scope / follow-ups

  • Test config under _dev/test/... (the package currently has none — separate effort).
  • The apiFormatters map in elastic/kibana's private_formatters/formatters.ts — needs to be authored against the var names declared here, not by reusing browserFormatters.

Made with Cursor

@shahzad31 @cursoragent
[Synthetics] Add synthetics/api input + api / api.network data …
bbfa853 
…streams

Adds the Fleet integration package side of API Journey monitors so private
locations can deliver `monitor.type: api` policies to Heartbeat. Companion
to elastic/synthetics#997 (agent), elastic/beats#50802 (Heartbeat input),
and elastic/kibana#270874 (UI / SO / project monitor).

WHY: elastic/kibana#270874 has a TODO in `format_synthetics_policy.ts`
explicitly waiting on this package — `inputs.find(input.type ===
'synthetics/api')` resolves to undefined until a `synthetics >= 1.7.0`
package is installed, so private-location API monitors return
`hasInput: false` and surface a "package needs upgrade" warning.
That PR also explicitly says the new input must be designed fresh, NOT
forked from `synthetics/browser`, because the var schema differs.

WHAT:

* manifest.yml: bump 1.6.1 -> 1.7.0 and add `synthetics/api` input.

* New data stream `api` (dataset `api`) — journey/step events for API
  monitors. ILM 30d hot / 365d delete (matches browser). Vars are the
  API-correct subset of browser:
    - drops `screenshots`, `throttling.config`, `synthetics_args` because
      `(*SourceJob).extraArgs` in elastic/beats#50802 strips
      `--sandbox`, `--screenshots`, `--throttling`/`--no-throttling` for
      `monitor.type: api`.
    - keeps `playwright_options` and `ignore_https_errors` (both supported
      by Playwright's APIRequestContext).
    - default `type: api`.
  Fields mirror browser, with `monitor.type: api`, a new
  `synthetics.journey.type` keyword (emitted by `synthexec/synthtypes.go`
  in elastic/beats#50802 and `JSONReporter` in elastic/synthetics#997),
  and the Chromium-only `browser.experience` / `browser.relative_trace`
  groups removed.

* New data stream `api_network` (dataset `api.network`) — the dataset
  `synthexec/enrich.go` in elastic/beats#50802 routes API-journey
  `journey/network_info` events to. ILM 1d hot / 14d delete (matches
  browser_network). Fields mirror browser_network (TLS / server / resolve
  groups already covered by ECS) with the same `journey.type` addition.
  No stream-level vars — inherits from the parent `synthetics/api` input,
  same model as `browser_network`.

Validated locally with `elastic-package format` and
`elastic-package check` (lint + build green).

Co-authored-by: Cursor <cursoragent@cursor.com>
@andrewkroh andrewkroh added the Integration:synthetics Elastic Synthetics label May 25, 2026
@github-actions

github-actions Bot commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Elastic Docs Style Checker (Vale)

Summary: 3 suggestions found

💡 Suggestions (3): Optional style improvements. Apply when helpful.
File Line Rule Message
packages/synthetics/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.
packages/synthetics/data_stream/api/manifest.yml 108 Elastic.WordChoice Consider using 'deactivate, deselect, hide, turn off' instead of 'disable', unless the term is in the UI.
packages/synthetics/data_stream/api/manifest.yml 172 Elastic.WordChoice Consider using 'deactivate, deselect, hide, turn off' instead of 'disable', unless the term is in the UI.

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

@elasticmachine

elasticmachine commented Jun 2, 2026

Copy link
Copy Markdown

💚 Build Succeeded

History

@botelastic

botelastic Bot commented Jul 2, 2026

Copy link
Copy Markdown

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

@botelastic botelastic Bot added the Stalled label Jul 2, 2026
@botelastic

botelastic Bot commented Jul 8, 2026

Copy link
Copy Markdown

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

Integration:synthetics Elastic Synthetics Stalled

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@shahzad31 @elasticmachine @andrewkroh